Digital image processing apparatus and method of controlling the same

ABSTRACT

A digital image processing apparatus and a method of controlling the same. The digital image processing apparatus includes: a display controller for displaying first content on a display unit; and an image generator for generating second content based on a photographing signal input, which may be from a user, and third content related to the second content.

CROSS-REFERENCE TO RELATED PATENT APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/110,395, filed Aug. 23, 2018, which is a continuation of U.S. patentapplication Ser. No. 14/874,976, filed Oct. 5, 2015, now U.S. Pat. No.10,079,041, which is a continuation of U.S. patent application Ser. No.13/739,275, filed Jan. 11, 2013, now U.S. Pat. No. 9,167,163, whichclaims the benefit of Korean Patent Application No. 10-2012-0005276,filed on Jan. 17, 2012, in the Korean Intellectual Property Office, thedisclosures of which are incorporated herein in their entirety byreference.

BACKGROUND

Disclosed herein are a digital image processing apparatus and a methodof controlling the same.

Digital image processing apparatuses, such as digital cameras andcamcorders, have become portable due, in part, to the miniaturizationand development of batteries, etc., and thus images are easily capturedanywhere. In addition, the digital image processing apparatuses providevarious functions for a user to capture a high quality image, even ifthe user is not an expert.

Also, the digital image processing apparatuses provide various functionsfor editing an image while capturing the image or after capturing theimage so that the user may generate an image having a desired form.

SUMMARY

One or more embodiments of the invention provide a digital imageprocessing apparatus capable of variously manufacturing additionalcontent related to a moving image, and a method of controlling thedigital image processing apparatus.

According to an embodiment of the invention, there is provided a digitalimage processing apparatus including: a display controller fordisplaying first content on a display unit; and an image generator forgenerating second content based on a photographing signal input, whichmay be input from a user, and third content related to the secondcontent.

The third content may be a moving image generated for a predeterminedtime based on a point of time when the photographing signal is applied.

The predetermined time may include an interval before the point of timewhen the photographing signal is applied.

The third content may include a pointer indicating a start time and anend time of the first content.

Final generation of the third content may be determined when displayingof the first content is ended.

The first content may be a moving image captured in real-time.

The first content may be a stored moving image.

Resolution of the third content may be below or equal to resolution ofthe second content.

The display controller may simultaneously reproduce the third contentwhile reproducing the second content.

When the second content is reproduced, the display controller may firstreproduce the third content, and reproduce the second content afterreproducing of the third content is ended.

According to another embodiment of the invention, there is provided amethod of controlling a digital image processing apparatus, the methodincluding: displaying first content; generating second content based ona photographing signal input, which may be input from a user; andgenerating third content related to the second content.

The generating of the third content may include generating a movingimage for a predetermined time based on a point of time when the secondcontent is generated.

The predetermined time may include an interval before a point of timewhen the photographing signal is applied.

The generating of the third content may include generating the thirdcontent such that the third content includes a pointer indicating astart time and an end time of the first content.

The method may further include determining final generation of thegenerated third content.

The first content may be a moving image.

Resolution of the third content may be below or equal to resolution ofthe second content.

According to another embodiment of the invention, there is provided amethod of controlling a digital image processing apparatus storingsecond content generated from first content constituting a moving image,and third content related to the second content, the method includingreproducing one of the second and third contents selected by a user.

The third content may be simultaneously reproduced while reproducing thesecond content.

The reproducing may include, when the second content is reproduced,first reproducing the third content and then reproducing the secondcontent after the reproducing of the third content is ended.

The second content may be a still image and the third content may be amoving image.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features and advantages of the invention will becomemore apparent by describing in detail exemplary embodiments thereof withreference to the attached drawings in which:

FIG. 1 is a block diagram of a digital image processing apparatusaccording to an embodiment of the invention;

FIG. 2 is a block diagram of a central processing unit/digital signalprocessor (CPU/DSP) according to an embodiment of the invention;

FIG. 3 is a block diagram for describing an image processing operationin the CPU/DSP of FIG. 2;

FIG. 4 is a block diagram for describing a relationship between inputand output images of an image signal processor, according to anembodiment of the invention;

FIG. 5 is a timing diagram for describing a method of generatingcontent, according to an embodiment of the invention;

FIG. 6 is a timing diagram for describing a method of generatingcontent, according to another embodiment of the invention;

FIG. 7 is a timing diagram for describing a method of generatingcontent, according to another embodiment of the invention;

FIG. 8 is a block diagram for describing content generated according toan embodiment of the invention;

FIG. 9 is a pictorial diagram of a reproduction image of contentgenerated according to an embodiment of the invention;

FIGS. 10 and 11 are flowcharts illustrating a method of controlling adigital image processing apparatus, according to an embodiment of theinvention;

FIG. 12 is a flowchart illustrating a method of controlling a digitalimage processing apparatus, according to another embodiment of theinvention;

FIG. 13 is a flowchart illustrating a method of controlling a digitalimage processing apparatus, according to another embodiment of theinvention;

FIG. 14 is a flowchart illustrating a method of controlling a digitalimage processing apparatus, according to another embodiment of theinvention; and

FIG. 15 is a flowchart illustrating a method of controlling a digitalimage processing apparatus, according to another embodiment of theinvention.

DETAILED DESCRIPTION

As the invention allows for various changes and numerous embodiments,particular embodiments will be illustrated in the drawings and describedin detail in the written description. However, this is not intended tolimit the invention to particular modes of practice, and it will to beappreciated that all changes, equivalents, and substitutes that do notdepart from the spirit and technical scope of the invention areencompassed in the invention. In the description of the invention,certain detailed explanations of related art are omitted when it isdeemed that they may unnecessarily obscure the essence of the invention.

The terms used in the present specification are merely used to describeparticular embodiments, and are not intended to limit the invention. Anexpression used in the singular encompasses the expression of theplural, unless it has a clearly different meaning in the context. In thepresent specification, it is to be understood that the terms such as“including” or “having,” etc., are intended to indicate the existence ofthe features, numbers, steps, actions, components, parts, orcombinations thereof disclosed in the specification, and are notintended to preclude the possibility that one or more other features,numbers, steps, actions, components, parts, or combinations thereof mayexist or may be added.

Those components that are the same or are in correspondence are renderedthe same reference numeral regardless of the figure number, andredundant explanations are omitted.

FIG. 1 is a block diagram of a digital image processing apparatus 1according to an embodiment of the invention.

Referring to FIG. 1, the digital image processing apparatus 1 mayinclude a photographing unit 100, an analog signal processor 110, acentral processing unit/digital signal processor (CPU/DSP) 120, a randomaccess memory (RAM) 130, a memory controller 140, a memory 150, aread-only memory (ROM) 160, a display driver 170, a display unit 171,and a manipulation unit 180.

Overall operations of the digital image processing apparatus 1 arecontrolled by the CPU/DSP 120. The CPU/DSP 120 provides a control signalfor operating each element to a lens driver 102, an iris driver 104, animage pickup device controller 106, or the like. In the currentembodiment, a CPU performing a control function and a DSP performing asignal process are configured as one device, but alternatively, the CPUand the DSP may be configured as individual devices.

The photographing unit 100 is an element for generating an image of anelectric signal from an incident light. The photographing unit 100 mayinclude a lens 101, the lens driver 102, an iris 103, the iris driver104, an image pickup device 105, the image pickup device controller 106,etc.

The lens 101 may include a plurality of groups of lenses or a pluralityof lenses. A location of the lens 101 may be adjusted by the lens driver102. The lens driver 102 adjusts the location of the lens 101 accordingto a control signal provided by the CPU/DSP 120.

An opening/closing degree of the iris 103 is adjusted by the iris driver104, and the iris 103 adjusts an amount of light incident on the imagepickup device 105. The iris driver 104 drives the iris 103 according toa control signal provided by the CPU/DSP 102.

The image pickup device 105 generates an image signal by capturing animage light of a subject, which has passed through the lens 101 and theiris 103. The image pickup device 105 may include a plurality ofphotoelectric conversion devices arranged in a matrix form, a chargetransmission path for transferring a charge from the photoelectricconversion device, etc. The image pickup device 105 may be a chargecoupled device (CCD) image sensor or a complementary metal oxidesemiconductor (CIS) image sensor.

Sensitivity or the like of the image pickup device 105 may be adjustedby the image pickup device controller 106. The image pickup devicecontroller 106 may control the image pickup device 105 according to acontrol signal automatically generated by an image signal input inreal-time, or a control signal manually input by a user.

An exposure time of the image pickup device 105 is adjusted by a shutter(not shown). Examples of the shutter include a mechanical shutter thatadjusts incidence of light by moving a mask, and an electronic shutterthat adjusts exposure by providing an electric signal to the imagepickup device 105.

The image pickup device controller 106 generates and applies a timingsignal to the image pickup device 105, and thus controls an imagingoperation of the image pickup device 105. Also, the image pickup devicecontroller 106 controls the image pickup device 105 to sequentially readimage signals when accumulation of charges in each scan line of theimage pickup device 105 is completed.

The analog signal processor 110 performs a noise reduction process, gainadjustment, waveform standardization, an analog-to-digital conversionprocess, or the like on an analog image signal supplied from the imagepickup device 105. A signal processed by the analog signal processor 110may be input to the CPU/DSP 120 through or not through the RAM 130.

The RAM 130 operates as a main memory of the digital image processingapparatus 1, and temporarily stores information required while operatingthe CPU/DSP 120. The ROM 160 stores a program, such as an operationsystem or an application system, for operating the digital imageprocessing apparatus 1.

The memory controller 140 controls input and output of data to thememory 150. The memory 150 may store a file, such as a still image or amoving image. The memory 150 may separately store a dual capture imageconstituting second content generated with respect to a main movingimage constituting first content, and a moving image highlightconstituting third content generated with the dual capture image. Inaddition, the memory 150 may store metadata about related informationbetween the primary through third contents in a separate file.

The display driver 170 controls an image output to the display unit 171.

The display unit 171 may display a live view image captured inreal-time, a quick view image temporarily displayed after image capture,a reproduction image of stored images, or the like. Also, the displayunit 171 may display various pieces of setup information.

The display unit 171 and the display driver 170 may be respectively aliquid crystal display (LCD) and an LCD driver. Alternatively, thedisplay unit 171 and the display driver 170 may be respectively anorganic light emitting display (OLED) and an OLED driver. Also, a touchscreen for recognizing a touch input may be provided to the display unit171, as the manipulation unit 180.

The manipulation unit 180 is a unit for generating a control signal byreceiving various commands from the user to manipulate the digital imageprocessing apparatus 1. The manipulation unit 180 may include variousinput keys, such as a power supply button, a shutter release button, azoom button, a mode dial, a menu button, a four-way button, and a jogdial. Also, the manipulation unit 180 may include a touch screen thatdetects contact of a body part of the user and generates a commandaccording to the contact.

The CPU/DSP 120 processes an input image signal, and controls eachelement according to the processed input image signal or an externalinput signal. The CPU/DSP 120 will be described in detail below withreference to FIGS. 2 and 3.

FIG. 2 is a block diagram of the CPU/DSP 120 according to an embodimentof the invention.

Referring to FIG. 2, the CPU/DSP 120 may include an image signalprocessor (ISP) 121, a moving image generator 122, a still imagegenerator 123, a display controller 124, a controller 125, and a timer126.

The ISP 121 reduces noise in input image data, and may perform imagesignal processes for improving image quality, such as gamma correction,color filter array interpolation, color matrix processing, colorcorrection, and color enhancement.

The ISP 121 may further perform a noise reduction process, a colorprocess, a blur process, an edge enhancement process, an imageinterpretation process, an image recognition process, and an imageeffect process. Examples of the image recognition process include a facerecognition process and a scene recognition process. In addition, theISP 121 may perform a display image signal process for displaying animage on the display unit 171. For example, the ISP 121 may performluminance level adjustment, color correction, contrast adjustment,outline enhancement adjustment, a screen division process, characterimage generation, and an image composition process.

The digital image processing apparatus 1 may be connected to an externalmonitor, and the ISP 121 may perform a predetermined image signalprocess such that an image is displayed on the external monitor.Processed image data is transmitted to the external monitor, and acorresponding image may be displayed on the external monitor.

The moving image generator 122 may generate a file of a moving image byperforming an image signal process for improving image quality andcompressing generated image data. Also, the moving image generator 122may restore moving image data from the file of the moving image. Animage may be compressed in a lossey or lossless manner.

Similarly, the still image generator 123 may generate a file of a stillimage by performing an image signal process for improving image quality,and compressing generated image data. Also, the still image generator123 may restore still image data from the file of the still image. Animage may be compressed in a lossey or lossless manner.

For example, a still image may be converted to a Joint PhotographicExperts Group (JPEG) format or a JPEG 200 format. Also, when a movingimage is recorded, a file of a moving image may be generated bycompressing a plurality of frames according to the Moving PictureExperts Group (MPEG) standard. An image file may be generated accordingto the Exchangeable image file format (Exif) standard.

In the current embodiment, the moving image generator 122 and the stillimage generator 123 are configured as individual units, butalternatively, they may be configured as one unit, and may perform asuitable operation according to a type of input image.

The display controller 124 controls the display driver 170 to control animage display on the display unit 171. For example, the displaycontroller 124 may control the display driver 170 such that a movingimage that is being captured or a moving image stored in the memory 150is displayed on the display unit 171. Also, while a still image is beingreproduced, the display controller 124 may control reproducing of arelated moving image highlight.

The controller 125 controls each element in the CPU/DSP 120. Thecontroller 125 may execute a program stored in the ROM 160, or maygenerate a control signal for controlling auto focusing, zoom change,focus change, or automatic exposure correction by including a separatemodule. The controller 125 provides the generated control signal to thelens driver 102, the iris driver 104, and the image pickup devicecontroller 106, and may control overall operations of elements includedin the digital image processing apparatus, such as a shutter and astrobe.

Also, the controller 125 may control each element such that the elementperforms an operation according to a signal applied via manipulation ofthe manipulation unit 180.

The timer 126 provides time information. The time information providedby the timer 126 may be inserted into a header region of a filegenerated by the moving image generator 122 or the still image generator123. Also, the time information may be used as a pointer for indicatingwhen a dual capture image is captured from a main moving image.Similarly, the time information may be used as a start pointer and anend pointer for indicating when a moving image highlight is generatedwith respect to a time interval of a main moving image.

Image data output from the ISP 121 is input to the memory controller 140directly or through the RAM 130. The memory controller 140 stores theimage data in the memory 150 automatically or according to a signal fromthe user. Also, the memory controller 140 reads data about an image froman image file stored in the memory 150, and inputs the read data to thedisplay driver 170 through the RAM 130 and the display controller 124 orthrough another path so that an image is displayed on the display unit171. The memory 150 may be detachable or permanently installed in thedigital image processing apparatus 1.

FIG. 3 is a block diagram for describing an image processing operationin the CPU/DSP 120 of FIG. 2. Only elements required for describing theimage processing operation are shown in FIG. 3.

Before describing FIG. 3, contents generated by the digital imageprocessing apparatus 1, according to embodiments of the invention willbe described.

The digital image processing apparatus 1 may generate a file of a movingimage by capturing a moving image according to a moving image capturingmode from among capturing modes. When the user applies a capture signalfor capturing a still image at a predetermined point of time whilecapturing a moving image, a still image is generated separately from themoving image. In other words, a dual capture image is generated assecond content with respect to a main moving image constituting firstcontent. Also, when the capture signal is applied to generate a dualcapture image, the digital image processing apparatus 1 separatelygenerates a moving image during a predetermined time interval based on apoint of time when the capture signal is applied. Such a moving image isa moving image highlight constituting third content related to thesecond content.

The digital image processing apparatus 1 may reproduce a moving imagealready captured and stored. As described above, a dual capture imageconstituting second content and a moving image highlight constitutingthird content may be generated with respect to the reproduced movingimage.

Generating of the main moving image, the dual capture image, and themoving image highlight described above will now be described in detail.

Referring to FIG. 3, a sensor output is input to the ISP 121 of theCPU/DSP 120. Here, the sensor output is an image signal which is outputfrom the image pickup device 105 and on which various signal processesand conversion to a digital signal are performed by the analog signalprocessor 110.

As described above, the ISP 121 performs various image signal processes.Also, the ISP 121 performs scale down on the sensor output to ensure thesensor output is a standard size.

FIG. 4 is a diagram for describing a relationship between input andoutput images of the ISP 121, according to an embodiment of theinvention. An ISP output that is output from the ISP 121 is smaller thanthe sensor output. An image signal output from the image pickup device105 is an actual file, and thus is larger in size than a stored movingimage. Accordingly, the ISP 121 scales-down the sensor output to astandard size. The standard size may be full high definition (HD), HD,or standard definition (SD).

Meanwhile, a dual capture image may be generated according to control ofthe user, for example, as the user presses a shutter release button.Here, the dual capture image is a still image. A size of the dualcapture image may be the same as the size of the sensor output accordingto a setting. Accordingly, the ISP 121 temporarily stores the sensoroutput on which scale down is not performed, and may use the storedsensor output when a dual capture image needs to be generated. The ISP121 may include a buffer memory for storing the sensor output.Alternatively, the sensor output may be temporarily stored in the RAM130.

The ISP output that is processed and output by the ISP 121 istransmitted to the moving image generator 122. The moving imagegenerator 122 generates a moving image stream by using the ISP outputreceived as a Codec, and generates a file of a moving image by using thegenerated moving image stream. Here, the generated file of the movingimage is a main moving image constituting first content.

Also, when a shutter release button is pressed, i.e., when a dualcapture command is applied, the ISP 121 transmits the sensor output infull size that is stored in the buffer memory or the like to the stillimage generator 123. Of course, the sensor output in the full sizetransmitted to the still image generator 123 is also an image signal onwhich various image signal processes are performed by the ISP 121.

The still image generator 123 generates a file of a still image by usingthe ISP output received as a Codec. Here, the generated file of thestill image is a dual capture image constituting second content.

In addition, when the dual capture command is applied, the moving imagegenerator 122 according to the current embodiment generates a separatefile of a moving image during a predetermined time interval based on apoint of time when the dual capture command is applied. In other words,the moving image generator 122 generates a moving image highlightconstituting third content.

Here, the moving image highlight may be generated while capturing themain moving image. If the predetermined time interval includes aninterval before the point of time when the dual capture command isapplied, the ISP 121 or the moving image generator 122 may include abuffer memory for storing the ISP output for a predetermined time.Accordingly, a frame image before the point of time when the dualcapture command is applied may be included in the moving imagehighlight.

Alternatively, the moving image highlight may be generated whencapturing of the main moving image is ended. The moving image highlightmay be generated by extracting a required region from the main movingimage by using time information on the predetermined time interval. Inthis case, a separate buffer memory may not be required even if thepredetermined time interval includes the interval before the point oftime when the dual capture command is applied.

The generated moving image highlight may be finally determined in termsof whether to be generated as a file when the capturing of the mainmoving image is ended. If the user determines that the moving imagehighlight is not required, the generated moving image highlight may bedeleted. Alternatively, if the user determines that the moving imagehighlight is required, the moving image highlight is finally generatedand stored as a file of a moving image.

In order to display an image being captured on the display unit 171, theISP 121 may transmit the ISP output to the display controller 124.

The display controller 124 transmits the ISP output to the displaydriver 170, and the display driver 170 may display the ISP output on thedisplay unit 171.

FIG. 5 is a timing diagram for describing a method of generatingcontent, according to an embodiment of the invention. In FIG. 5, amoving image capture operation is started as a shutter release button ispressed in a moving image capturing mode.

Referring to FIG. 5, the image pickup device 105 and the moving imagegenerator 122 capture a frame image in a predetermined period, forexample, in a 1/60 seconds interval, according to the moving imagecapture operation. A moving image stream is generated by using thecaptured frame image, and when the moving image capture operation isended, a file of a main moving image is generated by using the movingimage stream.

When a shutter release button is pressed at a predetermined point oftime, a dual capture image is generated. The dual capture image is astill image, and a size of a sensor output may be used as it is. Inother words, the dual capture image may have higher resolution than amain moving image or a moving image highlight. Accordingly, the sensoroutput, which is temporarily stored in the ISP 121 and on which scaledown is not performed, is used as the dual capture image.

Here, in order to classify a dual capture command from a main movingimage capture start and end command, for example, the dual capturecommand may be performed by half-pressing the shutter release button,and the main moving image capture start and end command may be performedby fully pressing the shutter release button. In the current embodiment,only the shutter release button is used, but alternatively, the dualcapture command and the main moving image capture start and end commandmay be performed by using different buttons.

Also, an operation for capturing a moving image highlight is performedby using the dual capture command. In the current embodiment, the movingimage highlight includes a predetermined time interval Ta before andafter the point of time when the dual capture command is applied. Thetime interval Ta may be pre-determined, or set or changed by the user.

As such, not only the dual capture image constituting a still image, butalso the moving image highlight related to the dual capture image aregenerated according to the dual capture command while capturing a movingimage.

FIG. 6 is a timing diagram for describing a method of generatingcontent, according to another embodiment of the invention.

Looking at a difference between the methods of FIGS. 5 and 6, a movingimage highlight of FIG. 6 only includes a time interval Tb before apoint of time when a dual capture command is applied.

FIG. 7 is a timing diagram for describing a method of generatingcontent, according to another embodiment of the invention.

Looking at a difference between the methods of FIGS. 5 and 7, a movingimage highlight of FIG. 7 only includes a time interval Tc after a pointof time when a dual capture command is applied.

FIG. 8 is a diagram for describing content generated according to anembodiment of the invention.

Referring to FIG. 8, a main moving image is generated as first content,and dual capture images S1 through Sn are generated as second contentsrelated to the main moving image. Also, while generating the dualcapture images S1 through Sn, moving image highlights M1 through Mn arerespectively generated as third contents related to the dual captureimages S1 through Sn.

A main moving image, a dual capture image, and a moving image highlightmay be stored as individual files. Also, it may be finally determinedwhether to generate the moving image highlight at a time when thecapturing of the main moving image is ended. In other words, only thedual capture image may be generated, and the moving image highlightrelated to the dual capture image may not be generated. Generation ofthe moving image highlight may be individually determined according tomoving image highlights.

As described above, the digital image processing apparatus 1 accordingto the current embodiment may generate a still image having a largersize (or higher resolution) than a moving image at a desired point oftime while capturing the moving image, and may separately generate andreproduce a moving image highlight related to the still image.

FIG. 9 is a diagram of a reproduction image of content generatedaccording to an embodiment of the invention.

Referring to FIG. 9, when a stored moving image is reproduced, a movingimage screen 900 is displayed on the display unit 171. Also, thumbnailimages 901 through 903 respectively of dual capture images related to amoving image may also be displayed on a region, for example, at thebottom of the display unit 171.

A state bar 910 for indicating a current reproduction location fromamong an entire moving image, may be displayed on the display unit 171.A thumbnail image of a corresponding dual capture image may be disposedat a location where the dual capture image is captured on the state bar910. The location where the dual capture image is captured is a temporallocation, and the location of the dual capture image may be determinedby using a pointer.

Also, when a moving image highlight related to the dual capture image isgenerated, the state bar 910 may display a time interval including themoving image highlight. A location of display of the time interval maybe determined by using start pointers SP1 through SP3 and end pointersEP1 through EP3.

A pointer or metadata including a start pointer and an end pointer willnow be described in detail.

In order to display a dual capture image and a moving image highlight,or information related to a main moving image as shown in FIG. 9 whilereproducing the main moving image, it is required to record that imagesare related to each other. This is because a corresponding still imagecan be linked while reproducing a moving image when still images storedwhile capturing a corresponding moving image are identified.

Accordingly, the metadata should include file information about a mainmoving image, a dual capture image, and a moving image highlight. Here,the file information may be file names of the main moving image, thedual capture image, and the moving image highlight.

Also, the metadata includes time information about points of time whenthe main moving image, the dual capture image, and the moving imagehighlight are captured. The time information of the main moving image isinformation about a moving image capture start time. The timeinformation of the dual capture image is information about a framenumber in a moving image or a point of time when a dual capture commandis applied. The time information of the moving image highlight is startand end times of the moving image highlight. Alternatively, the timeinformation of the moving image highlight may be a frame number in themain moving image in the start and end times.

The file information and the time information may be managed by beingincluded in the metadata, and the metadata may be managed using any oneof various methods.

The metadata may be managed by storing the metadata in a header of themain moving image as extension metadata. Alternatively, the metadata maybe included in a header of the dual capture image or moving imagehighlight. Alternatively, metadata in different ranges may be stored ineach content. For example, metadata including all information about themain moving image, the dual capture image, and the moving imagehighlight may be recorded in the header of the main moving image,metadata including information about the dual capture image and themoving image highlight may be recorded in the header of the dual captureimage, and metadata including information about the moving imagehighlight and the dual capture image may be recorded in the header ofthe moving image highlight.

Alternatively, the metadata may be managed by storing the metadata as ametafile separate from each content. Here, the metafile may be in onefile including all metadata of the main moving image, the dual captureimage, and the moving image highlight. Alternatively, the metafile maybe generated and stored separately according to main moving images. Inother words, each metafile may only include information about a dualcapture image and a moving image highlight generated in relation to acorresponding main moving image.

FIGS. 10 and 11 are flowcharts illustrating a method of controlling adigital image processing apparatus 1 of FIG. 1, according to anembodiment of the invention.

Referring to FIG. 10, when a capturing mode is started, periodicalimaging is performed in the image pickup device in operation S100. Animage signal generated via the periodical imaging is processed and thendisplayed on the display unit 171 as a live view image, in operationS101.

It is determined whether there is a shutter input by a user, i.e.,whether a shutter release button is pressed in operation S102. If it isdetermined that there is no shutter input, operation S100 is performedto display the live view image. On the other hand, if it is determinedthat there is a shutter input, it is determined whether the digitalimage processing apparatus 1 is in a moving image capturing mode inoperation S103.

If it is determined that the digital image processing apparatus 1 is notin the moving image capturing mode, it is determined that the digitalimage processing apparatus 1 is in a still image capturing mode, andthus auto focus (AF) and auto exposure (AE) operations are performed inoperation S104, and an image is captured in operation S105. Then,various image signal processes are performed on the captured image, andthen a file of a still image is generated in operation S106.

On the other hand, if it is determined that the digital image processingapparatus 1 is in the moving image capturing mode, a moving imagecapture operation is performed.

Referring to FIG. 11, when the moving image capture operation isstarted, a moving image stream is generated by using an ISP output inoperation S200. Operation S200 is continuously performed until themoving image capture operation is ended.

It is determined whether a dual capture command is applied by a userwhile generating the moving image stream in operation S201, and if thereis no dual capture command, operation S200 and operation S202 ofdetermining whether to end the moving image capture operation arerepeatedly performed.

On the other hand, if the dual capture command is applied whilegenerating the moving image stream, a dual capture image is generated inoperation S203. Then, a moving image highlight related to the dualcapture image is generated in operation S204.

It is determined whether the moving image capture operation is ended inoperation S205, and if the moving image capture operation is not ended,operations S200 through S204 are repeated. If it is determined that themoving image capture operation is ended in operation S205, it isdetermined whether to store the moving image highlight generated inrelation to the dual capture image in operation S206.

If the moving image highlight is determined to be stored by the user,files of the main moving image, the dual capture image, and the movingimage highlight are each generated and stored in operation S207. On theother hand, if the moving image highlight is determined to be not storedby the user, files of the main moving image and the dual capture imageare each generated and stored in operation S208.

In the current embodiment, the storing of the moving image highlight isdetermined after the capturing of the main moving image is ended, butalternatively, files of all moving image highlights may be automaticallygenerated and stored without a decision of the user. Alternatively, thegeneration of files of the moving image highlights may be determined ina batch or individually. Also, the generation of files may be determinednot only with respect to the moving image highlight but also the dualcapture image.

FIG. 12 is a flowchart illustrating a method of controlling the digitalimage processing apparatus 1 of FIG. 1, according to another embodimentof the invention.

Referring to FIG. 12, a moving image reproduction operation is performedby extracting and displaying a moving image stream from a selected fileof a main moving image in operation S300. A moving image is continuouslyreproduced until the moving image reproduction operation is ended.

While reproducing the moving image, it is determined whether a dualcapture command is applied by a user in operation S301, and if the dualcapture command is not applied, operation S300 and operation S302 ofdetermining whether to end the moving image reproduction operation arerepeatedly performed.

If the dual capture command is applied while reproducing the movingimage, a dual capture image is generated by extracting a frame imagefrom the moving image stream in operation S303. Here, a maximumresolution of the dual capture image is a resolution of the moving imagebeing reproduced because unlike a dual capture image generated whilecapturing a moving image, an obtainable maximum resolution of the dualcapture image generated at this time is the resolution of the movingimage being reproduced.

Next, a moving image highlight related to the dual capture image isgenerated by extracting an interval of the moving image stream beingreproduced in operation S304.

It is determined whether the moving image reproduction operation isended in operation S305, and if not, operations S300 through S304 arerepeated. If it is determined that the moving image reproductionoperation is ended in operation S305, it is determined whether to storethe moving image highlight generated by the dual capture command inoperation S306.

If it is determined to store the moving image highlight by the user,files of the dual capture image and the moving image highlight areadditionally respectively generated and stored in operation S307. On theother hand, if it is determined not to store the moving image highlightby the user, only a file of the dual capture image is additionallygenerated and stored in operation S308.

In the current embodiment, the storing of the moving image highlight isdetermined after the reproducing of the main moving image is ended, butalternatively, generating of files of the dual capture image and themoving image highlight may be determined in any method as describedabove.

FIG. 13 is a flowchart illustrating a method of controlling the digitalimage processing apparatus 1 of FIG. 1, according to another embodimentof the invention.

Referring to FIG. 13, when a still image is selected by a user inoperation S400, it is determined whether a moving image highlightrelated to the selected still image exists in operation S401.

If it is determined that the moving image highlight related to theselected still image as a dual capture image exists, the still image andthe moving image highlight are simultaneously reproduced in operationS402. For example, the still image may be reproduced throughout ascreen, and the moving image highlight may be reproduced in a small sizeon a partial area of the display unit 171. On the other hand, the movingimage highlight may be reproduced throughout a screen, and the stillimage may be reproduced in a small size on a partial area of the displayunit 171.

Meanwhile, if it is determined that the moving image highlight relatedto the selected still image does not exist in operation S401, only thestill image is reproduced in operation S403.

FIG. 14 is a flowchart illustrating a method of controlling the digitalimage processing apparatus 1 of FIG. 1, according to another embodimentof the invention. The method of FIG. 14 is different from the method ofFIG. 13 in terms of reproducing of a moving image highlight, and onlythis difference will be described.

Referring to FIG. 14, if it is determined that the moving imagehighlight related to the selected still image as a dual capture imageexists in operation S501, the moving image highlight is first reproducedin operation S502. Then, when the reproducing of the moving imagehighlight is ended, the selected still image is reproduced in operationS503.

FIG. 15 is a flowchart illustrating a method of controlling the digitalimage processing apparatus 1 of FIG. 1, according to another embodimentof the invention. In the current embodiment, a main moving image, ofwhich a dual capture image and a moving image highlight are generated,is selected as a moving image.

Referring to FIG. 15, the moving image is reproduced by extracting anddisplaying a moving image stream from a selected file of a main movingimage in operation S600. The moving image is continuously reproduceduntil a moving image reproduction operation is ended.

Then, information about a dual capture image and a moving imagehighlight related to the main moving image are displayed together inoperation S601. For example, the information may be a pointer indicatinga point of time when the dual capture image is captured from the mainmoving image being reproduced or a pointer indicating an interval wherethe moving image highlight is captured. The information may be displayedby extracting file information or time information from metadatarecorded on a header region of the main moving image or separatemetadata.

Then, it is determined whether the dual capture image displayed with themain moving image is selected by the user while reproducing the mainmoving image in operation S602. If the dual capture image is selected, astill image is reproduced as described with reference to FIGS. 13 and14. Although not illustrated, the dual capture image may be temporarilyreproduced, and then the main moving image may be reproduced again aftera predetermined time.

Also, it is determined whether the moving image highlight is selected bythe user while reproducing the main moving image, in operation S603. Ifthe moving image highlight is selected, the selected moving imagehighlight is reproduced in operation S604, and then it is determinedwhether to end the reproducing of the moving image in operation S605.Then, when the reproducing of the moving image highlight is ended, themain moving image is reproduced again.

If no content is selected in operations 602 and S603, the main movingimage is reproduced while determining whether the reproducing of themoving image is ended in operation S606. If it is determined that thereproducing of the main moving image is not ended, operation S600 isperformed.

As described above, the digital image processing apparatus 1 accordingto the current embodiment may generate a still image having a largersize (or higher resolution) than a moving image at a point of timedesired by the user while capturing the moving image, and separatelygenerate and reproduce a moving image highlight related to the stillimage.

According to the embodiments of the invention, the digital imageprocessing apparatus can satisfy diversified tastes of users byvariously manufacturing additional contents related to moving images.

The invention can also be embodied as computer readable codes on acomputer readable recording medium. The computer readable recordingmedium is any non-volatile data storage device that can store data whichcan be thereafter read by a computer system.

The computer readable codes are configured to perform operations forrealizing a method a method of controlling a digital image processingapparatus according to the invention when read and executed by aprocessor (for example, the CPU/DSP 120) from the computer readablerecording medium. The computer readable codes may be realized in variousprogramming languages. Also, functional programs, codes, and codesegments for accomplishing the invention can be easily construed byprogrammers skilled in the art to which the invention pertains.

Examples of the computer readable recording medium include read-onlymemory (ROM), random-access memory (RAM), CD-ROMs, magnetic tapes,floppy disks, optical data storage devices, etc. The computer readablerecording medium can also be distributed over network coupled computersystems so that the computer readable code is stored and executed in adistributed fashion.

All references, including publications, patent applications, andpatents, cited herein are hereby incorporated by reference to the sameextent as if each reference were individually and specifically indicatedas incorporated by reference and were set forth in its entirety herein.

The embodiments herein may be described in terms of functional blockcomponents and various processing steps. Such functional blocks may berealized by any number of hardware and/or software components thatperform the specified functions. For example, the described embodimentsmay employ various integrated circuit components, e.g., memory elements,processing elements, logic elements, look-up tables, and the like, whichmay carry out a variety of functions under the control of one or moremicroprocessors or other control devices. Functional aspects may beimplemented in algorithms that execute on one or more processors.Furthermore, the embodiments of the invention could employ any number ofconventional techniques for electronics configuration, signal processingand/or control, data processing and the like.

For the sake of brevity, conventional electronics, control systems,software development and other functional aspects of the systems (andcomponents of the individual operating components of the systems) maynot be described in detail. Furthermore, the connecting lines, orconnectors shown in the various figures presented are intended torepresent exemplary functional relationships and/or physical or logicalcouplings between the various elements. It should be noted that manyalternative or additional functional relationships, physical connectionsor logical connections may be present in a practical device. Moreover,no item or component is essential to the practice of the inventionunless the element is specifically described as “essential” or“critical”.

The use of “including,” “comprising,” or “having” and variations thereofherein is meant to encompass the items listed thereafter and equivalentsthereof as well as additional items. Unless specified or limitedotherwise, the terms “mounted,” “connected,” “supported,” and “coupled”and variations thereof are used broadly and encompass both direct andindirect mountings, connections, supports, and couplings. Further,“connected” and “coupled” are not restricted to physical or mechanicalconnections or couplings. Expressions such as “at least one of,” whenpreceding a list of elements, modify the entire list of elements and donot modify the individual elements of the list.

The steps of all methods described herein are performable in anysuitable order unless otherwise indicated herein or otherwise clearlycontradicted by context. The use of any and all examples, or exemplarylanguage (e.g., “such as”) provided herein, is intended merely to betterilluminate the invention and does not pose a limitation on the scope ofthe invention unless otherwise claimed. The words “mechanism” and“element” are used herein generally and are not limited solely tomechanical embodiments.

While this invention has been particularly shown and described withreference to embodiments thereof, it will be understood by those ofordinary skill in the art that various changes in form and details maybe made therein without departing from the spirit and scope of theinvention as defined by the appended claims. The embodiments should beconsidered in a descriptive sense only and not for purposes oflimitation. Therefore, the scope of the invention is defined not by thedetailed description of the invention but by the appended claims, andall differences within the scope will be construed as being included inthe invention.

What is claimed is:
 1. A digital image processing apparatus comprising:a display controller for displaying first content on a display unit; andan image generator for generating second content based on aphotographing signal input, and third content related to the secondcontent.
 2. The digital image processing apparatus of claim 1, whereinthe third content is a moving image generated for a predetermined timebased on a point of time when the photographing signal is applied. 3.The digital image processing apparatus of claim 2, wherein thepredetermined time comprises an interval before the point of time whenthe photographing signal is applied.
 4. The digital image processingapparatus of claim 1, wherein the third content comprises a pointerindicating a start time and an end time of the first content.
 5. Thedigital image processing apparatus of claim 1, wherein final generationof the third content is determined when displaying of the first contentis ended.
 6. The digital image processing apparatus of claim 1, whereinthe first content is a moving image captured in real-time.
 7. Thedigital image processing apparatus of claim 1, wherein the first contentis a stored moving image.
 8. The digital image processing apparatus ofclaim 1, wherein resolution of the third content is below or equal toresolution of the second content.
 9. The digital image processingapparatus of claim 1, wherein the display controller simultaneouslyreproduces the third content while reproducing the second content. 10.The digital image processing apparatus of claim 1, wherein, when thesecond content is reproduced, the display controller first reproducesthe third content, and reproduces the second content after reproducingof the third content is ended.
 11. A method of controlling a digitalimage processing apparatus, the method comprising: displaying firstcontent; generating second content based on a photographing signalinput; and generating third content related to the second content. 12.The method of claim 11, wherein the generating of the third contentcomprises generating a moving image for a predetermined time based on apoint of time when the second content is generated.
 13. The method ofclaim 12, wherein the predetermined time comprises an interval before apoint of time when the photographing signal is applied.
 14. The methodof claim 11, wherein the generating of the third content comprisesgenerating the third content such that the third content comprises apointer indicating a start time and an end time of the first content.15. The method of claim 11, further comprising determining finalgeneration of the generated third content.
 16. The method of claim 11,wherein the first content is a moving image.
 17. The method of claim 11,wherein resolution of the third content is below or equal to resolutionof the second content.
 18. A method of controlling a digital imageprocessing apparatus storing second content generated from first contentconstituting a moving image, and third content related to the secondcontent, the method comprising reproducing one of the second and thirdselected contents.
 19. The method of claim 18, wherein the third contentis simultaneously reproduced while reproducing the second content. 20.The method of claim 18, wherein the reproducing comprises, when thesecond content is reproduced, first reproducing the third content andthen reproducing the second content after the reproducing of the thirdcontent is ended.